Crane, Construction Machine Or Industrial Truck Simulator

ABSTRACT

A crane, a construction machine or an industrial truck, with a control station including at least one input means for inputting control commands, a graphical simulation module for calculating a virtual representation of the machine surroundings and/or machine components visible from the control station, such as a boom or a load hook, and a display device for displaying the calculated virtual representation, wherein a movement simulation module is provided for determining movements and/or deformations of the machine components according to the inputted control commands, depending on which the graphical simulation module calculates the virtual representation. Proposed is a data emulation using hardware components, which carry out actual actuating movements and thus simulate “actual” actuating movements of the machine to be simulated, in order to provide corresponding movement data more rapidly and with less computing performance, whereby a more realistic simulation can be achieved in real-time or almost real-time.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/069,384 filed 11 Jul. 2018, which is a § 371 national stage ofInternational Application PCT/EP2017/000024, with an internationalfiling date of 10 Jan. 2017, which claims the benefit of DE PatentApplication Serial No. 10 2016 000 353.7, filed on 14 Jan. 2016, thebenefit of the earlier filing date of which is hereby claimed under 35USC § 119(a)-(d) and (f). The entire contents and substance of allapplications are hereby incorporated by reference.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

THE NAMES OF THE PARTIES TO A JOINT RESEARCH AGREEMENT

Not Applicable

SEQUENCE LISTING

Not Applicable

STATEMENT REGARDING PRIOR DISCLOSURES BY THE INVENTOR OR A JOINTINVENTOR

Not Applicable

BACKGROUND OF THE DISCLOSURE 1. Field of the Invention

The present invention relates generally to simulator for a crane, aconstruction machine or an industrial truck, with a control stationhaving at least one input means for inputting control commands, agraphical simulation module for computing a virtual representation ofthe machine surroundings and/or machine components such as a boom orload hook visible from the control station, as well as a display devicefor displaying the computed virtual representation, wherein the movementsimulation module is provided to determine movements and/or deformationsof the machine components depending on the input control commands,depending on which the graphical simulation module computes the virtualrepresentation.

2. Description of Related Art

Cranes and similar large devices such as pile drivers, surface miners orcable excavators are very complex in operation and control, and thusdifficult to learn, so that common training materials such as photos,plans or even films are not sufficient to actually teach the operationand monitoring in a demonstrative manner and to make it easilyunderstandable. Here, not only the large number of control functions andtheir interplay as well as the involved input means, which are quitecomplex in their entirety, such as joysticks, foot pedals and controlswitches, are a problem, but also the unusual, machine-specificreactions of the machine structure to the movements of the actuators.Cranes such as tower slewing cranes or telescopic luffing cranes, butalso dock-side cranes or maritime cranes have large, slender structuralcomponents such as booms or jobs or tower structures, which can twistand are relatively soft, so that structural deformations and oscillatingor swinging movements go hand in hand with acceleration and decelerationprocesses of the actuators, which complicate safe operation even forexperienced crane operators when they change to a new type of crane. Incontrast to small devices with structures that are assumed to beapproximately rigid, there may be deformations of the tower structure orof the boom system for example in tower slewing cranes when a load ispicked-up, or the load can swing back when rotating the load about theupright axis and the boom may follow this swing accordingly. Similarthings can happen in cable excavators or pile drivers, so that the craneoperator or machine operator becomes uncertain when they put intopractice the control processes learned in theory, and learn thecorresponding crane reactions.

In order to have a more realistic training situation, the use of cranesimulators was proposed, in which the crane operator to be trained canenter control commands in an almost realistic control station, which maycorrespond to the crane operator's cab of a corresponding crane type,through input means provided there, such as joysticks, pedals, controlswitches or touchscreens, in order to learn the crane reactions to thesecontrol commands in a most realistic manner. To that end, a virtualrepresentation of the crane surroundings as well as the crane componentsvisible from the control station such as the boom and load hook arerepresented on a display device, which may include multiple monitors ina manner known per se, which are arranged in the field of view of thecontrol station, wherein the virtual representation of the cranesurroundings and the crane components is computed by a graphicalsimulation module depending on the input control commands.

If, for example, the crane is turned about the vertical axis, or acorresponding control command is input, the graphical simulation modulecomputes the representation of the crane surroundings in such a way thatthis representation moves, on the screen, from right to left, or viceversa, so that the virtual crane surroundings displayed on the displaydevice move past the crane operator in a manner similar to the “real”rotation of a crane in the crane cab thereof If, on the other hand, acontrol command is input that lowers the load hook and/or pivots theboom downward, the graphical simulation module changes the virtualrepresentation in such a way that the crane hook moves downward on thedisplay device and the boom is pivoted downward, respectively. Throughsuch a realistic simulation of crane operations, the crane operatoreasily gets a feeling for what the reactions to the actuations of theinput means of the control station might be.

Such a crane simulator is known from patent document DE 10 2013 011 818A1, for example. In this document, a crane operator's cab is provided asa control station having corresponding input means, wherein theinspection windows or the window panes of the simulated crane operator'scab are replaced by screens, on which the virtual representation of thecrane surroundings is displayed. Through a technical simulation module,also the dynamic behavior of the control and drive components is to besimulated and considered for the screen representation, wherein in thiscase, mainly the actuations of crane components, e.g. of the liftingunit, occurring in certain crane movements, are represented.

However, being close to reality is still limited in the above mentionedcrane simulator. On the one hand, a delayed response or a representationof the virtual reality which is delayed compared to the actual cranereactions occurring in real time may occur due to the complexcomputation processes required for the dynamic behavior, in particularif multiple actuations are to be employed virtually at the same time. Onthe other hand, the intuitive feeling for the crane reactions to certaincontrol commands is limited due to the representations of the virtualcrane surroundings that can be generated on the screens.

Against this background, the object underlying the present invention isto provide an improved simulator of the type described above, whichprevents disadvantages of the prior art and further develops the priorart in an advantageous manner. In particular, a more realisticsimulation of the crane or machine operation is to be achieved, whichimproves the training effect and teaches the actual crane or machinebehavior in a better way and makes it easier to learn.

SUMMARY OF THE INVENTION

According to the invention, this object is achieved by a simulator for acrane, a construction machine or an industrial truck, with a controlstation comprising at least one input means for inputting controlcommands, a graphical simulation module for calculating a virtualrepresentation of the machine environment and/or machine componentsvisible from the control station, as well as a display device fordisplaying the calculated virtual representation, wherein a movementsimulation module for determining movements and/or deformations ofmachine components such as the crane jib or the load hook depending onthe entered control commands is provided, and the graphical simulationmodule is configured for calculating the virtual representationdepending on the determined movements and/or deformations, characterizedin that the movement simulation module includes a data emulationapparatus for emulating movement data of the machine to be simulated,wherein the mentioned data emulation apparatus includes at least oneactuator component for performing an actuator movement depending on acontrol command entered at the control station and the graphicalsimulation device is configured to generate the virtual representationunder consideration of the performed actuator movement.

Thus, a data emulation using hardware components is proposed, whichperform actual actuations and thus simulate “real” actuations of themachine to be simulated, in order to provide corresponding movement datamore rapidly and with less computing power, whereby a more realisticsimulation can be achieved in real-time or almost real-time. Here, thesimulator cannot have the movement parameters required for the movementsimulation computed by a simulation computer, but at least partiallydetermine by way of data emulation using actually moving hardwarecomponents, which may be a part of the simulator. Such a data emulationmodule of the simulator can in particular include actuator componentsand/or power electronics components, by means of which actuations areactually performed, which simulate the real crane or machine movements,and these movements provide characterizing data, for example in the formof sensor signals, which represent the actuations of the mentioned drivecomponents. Due to such a data emulation, movement and/or positionparameters, which can be further employed for the movement simulation,can be provided significantly faster and with less computing power,which permits a more realistic simulation in real time or almost realtime.

In order to achieve a particularly fast, realistic determination ofmovements of the machine components depending on the control commandsinput at the control station, according to a further aspect, thementioned movement simulation module can be formed as a hybrid device orhybrid module, which, on the one hand, includes a computer for thesimulation of movement and/or position parameters, and on the other,hardware components such as drive units, rotary encoders or frequencyconverters, which are at least simulator to the real crane or machineactuators, and by means of which actuations are simulated and movementand/or position parameters are determined. In particular, “real”hardware components are used, which are also mounted as actuators and/orcontrol device components in the crane to be simulated or the machine tobe simulated.

In particular, the movement simulation module may include the controlcabinet or at least a part of the control cabinet or the componentsthereof, which is also used in the machine to be simulated and formspart of the machine control. In particular, power electronics and/or atleast part of the power electronics such as a frequency converted can beused to simulate the actuations triggered by control command input atthe control station.

Furthermore, in a further development of the invention, actuator units,for example in the form of servo motors, can be used, which serve forthe emulation of the actuations of the machine or the machine componentsto be simulated. Advantageously, one drive unit in the form of a servodrive unit, which—in particular via the above mentioned frequencyconverter—is controlled in accordance with a control command, is usedfor a respective actuator axis, and further advantageously, can becoupled with a further drive unit, for example in the form of a servodrive unit, by means of which a counter moment and/or a counter load canbe exerted in order to simulate actually occurring loads, resistances orinertia. For example, a load that counteracts a lifting mechanism can besimulated by means of the mentioned second drive unit, or a wind momentcan be simulated, which counteracts a drive or a rotary mechanism.

The actuation of the first-mentioned drive unit which is performed, asthe case may be, in consideration of the applied counter moment or theapplied counter load, can be detected by a suitable detection device,wherein a corresponding detection signal represents the actuallyachieved actuation and can be used as a sensor signal in the furthersimulation, in particular in order to determine movements and/orpositions and/or deformations of the structural parts in theabove-mentioned manner and/or to simulate the virtual representation ofthe machine surroundings and/or the machine components visible therein.

Advantageously, multiple such drive units or multiple such drive unitpairs including drive and counter load drive as well as in each case oneassigned detection device are used in order to be able to determine thedifferent actuator axis and the actuations of the machine operation tobe simulated performed in this regard.

Thus, according to a further aspect, sensor values of the drive units ofthe actuation axes, which are actuated and moved depending on thecontrol commands input at the control station are not simulated orcomputed by means of a computation model, but instead are emulated orsimulated by hardware components, which are most similar to the realactuator components of the machine to be simulated, and output as actualsensor values.

By such a data emulation system, the movement simulation module candetermine movements and/or positions of the machine components much morefaster and with less computing power, so that the virtual representationof the machine surroundings and/or of the machine components and alsothe involved actuations of the machine station can be achieved very muchfaster and realistic. In addition, the generated sensor signals can bedisplayed at the control station and/or be used for further monitoringmeasures such as bearing load monitoring or restriction of workingareas, which can be displayed and/or simulated at the control station.

In the case that the crane simulator is used to simulate a tower slewingcrane and the operation thereof, the above-mentioned drive unit pairsfor performing the corresponding actuations and provision of thecorresponding counter moment or counter load may correspond to the towerslewing crane—or in a top rotator the rotary mechanism of the boom, thelifting unit and the trolley drive.

According to a further aspect, the movement simulation module isconfigured in such a way that the crane or machine structure is notconsidered to be a rigid, so to say un-endlessly rigid structure, but asan elastically deformable and/or resilient and/or relatively softstructure, which—in addition to the actuation axis of the machine suchas the boom pivot axis or the tower slewing axis—permits movementsand/or position changes by deformations of the axis of rotation of thetower. The consideration of the movability of the machine structure as aconsequence of structural deformations under load or dynamic loads isimportant especially in elongated slender and structures that aredeliberately exploited in regard to the static and dynamic boundaryconditions—under consideration of the required safety—such as in cranes,since in this case, recognizable movement components, for example forthe crane operator's cab, but also the load hook position are added bythe deformations of the structural components. In order to enable anactually realistic training here, the movement simulation moduleaccounts for such deformations under static and dynamic loads.

In particular, the determination device may comprise a calculation unitfor the determination of such structural deformation, which calculatesthese structural changes based upon a stored calculation schemedepending on the control station. Such a model can be structuredsimilarly to a finite element model or be a finite element model,wherein advantageously, however, a model which is simplified comparedwith a finite element model is used, which can be determinedempirically, for example, by the detection of structural deformationsunder certain control commands and/or load conditions at the real craneor the real machine. Such a calculation model may operate with tables,in which certain deformations are assigned to certain control commands,wherein interim values of the control commands can be translated intocorresponding deformations by means of an interpolation device.

The use of such a computation scheme, which is simplified in comparisonto a finite element model, permits a faster determination of thestructural deformations and thus a more realistic simulation of machinemovements in real time or almost real time under less computation power.

The structural component deformations considered by the movementsimulation model can, on the one hand, be considered in the control ofthe drive device for moving the control station, so that the controlstation simulates the control station movements occurring due to thestructural deformations. Alternatively or additionally, the determinedstructural component deformations can be considered also in thecalculation of the virtual representation of the machine surroundingsand/or the machine components visible therein, for example in such a waythat the bending of the boom is illustrated in the virtualrepresentation, or the horizon of the crane surroundings is moved a bitupwards, in order to simulate a slight forward pitching of the craneoperator's cab due to a tower deformation, for example.

According to a further aspect, it is proposed to simulate crane ormachine reactions to control commands input at the control station notonly in the form of a virtual representation on the display device, butalso in an actual movement of the control station along with the craneor machine reaction, in order to get the dynamic machine reactionsacross to the user of the simulator in a more realistic manner. Thecontrol station, which may include an operator chair, for example, is nolonger static in space, or mounted on the ground, but moveable in spaceby a drive device.

In particular, the control station is moveably mounted and can be movedby a drive device depending on the movement determined by the movementsimulation module and/or deformation of the machine components. In theevent that the movement simulation module determines deflections ofmachine components such as the crane tower due to actuations ordeformations, which would influence the position of the real craneoperator's cab, the drive device is correspondingly controlled by adrive control device to simulate the movement of the crane operator'scab and move the control station correspondingly. If, for example, acommand for turning the crane about the vertical axis is input at thecontrol station, the control station is turned correspondingly about thevertical axis by the drive device. If, for example, a control commandfor the lifting of a heavy load is input, which in reality may lead to aslight forward pitching of the crane structure, the control station isdisplaced slightly forward and/or slightly tilted forward.

To enable a most realistic simulation of the control station movementsoccurring in real operation, the drive device can be formed to bemoveable along multiple axes and perform both rotary and translatormovements. In particular, the control station can be mounted in a mannerto be moveable along multiple axes, and the control station may includeat least one vertical axis of rotation and at least one horizontalpitching axis and/or two horizontally oriented translator axes. To beable to simulate complex control station movements as well, the drivedevice may comprise three rotary and pivoting axes, or be configured tobe operating in a three-axially rotational manner and be configured tobe operating in a three-axially translational manner, so that thecontrol station can be rotated or tilted about all three spatial axesand be displaced in all three spatial directions. Depending on the cranetype or machine type to be simulated, simpler configurations of thedrive device with fewer axes of motion can also be considered.

In order to further increase the reality feeling of the user of thesimulator, according to a further aspect, it is provided that thevirtual representations provided by the graphical simulation module fromthe simulation world are superimposed with live images from the controlstation, which may, for example, show movements of the user of thesimulator. In particular, the virtual representations of the machinesurroundings generated by the graphical simulation module and/or themachine components visible therein on the one hand, and live images of alive camera captured at the control station on the other hand, can berepresented on the display device simultaneously and in a superimposedmanner. Such a superposition of images from the simulation world andlive images provides the user of the simulator with a particular strongfeeling of reality.

Advantageously, to that end, a display device wearable on the head, inparticular in the type of glasses, can be used as a display device, forexample in the form of virtual reality glasses, and a camera, which isadvantageously also wearable on the head, for example formed as a helmetcamera or integrated in the virtual reality glasses, can be used, whichprovides the mentioned live images, which are displayed on the displaydevice, in particular the virtual reality glasses, together with theartificially generated virtual representation.

The mentioned camera for the provision of the live images canadvantageously be a stereoscopic camera, which provides stereoscopicimages preferably in the camera viewing direction which at leastapproximately equals the viewing direction of the eye pair of a user,which can be faded-in at a corresponding location of the display device,in particular the virtual reality glasses. As a result, a particularlystrong reality feeling of the user can be achieved.

Basically, however, it would also be possible to superimpose images fromthe simulation world and the mentioned live images on a per seconventional screen, wherein a user, for example, could wear a liveimage camera on the head which provides images at least approximatelycorresponding to the viewing direction of the user, so that a livecaptured user arm or the live-captured part of the control station canbe faded-in on the display device for example in the form of multiplescreen. However, a more realistic and thus more impressive simulationcan be achieved by superimposition on the visual surfaces of virtualreality glasses, however.

The superimposition device for superimposing the live images of thecamera with the virtual representation from the graphical simulationmodule can advantageously be operating according to the “green screentechnique”, wherein the superimposition device recognizes color surfacesof a certain color in the live image and replaces these image areas withthe virtual representation from the simulation module. To that end,advantageously, the control station may include an operator's cab wall,in which window areas—for example corresponding to the inspectionwindows of a real crane operator's cab—are colored in a key color, whichdiffers from the remaining colors of the remaining components located inthe camera's field of vision, such as the color of the window frame, theinput means and the operator clothes as well as skin color of theoperator in a most clear manner, so that the live image captured in thecontrol station shows the mentioned colored surfaces in a certain colorrepresentation, while all of the other image surfaces are shown in othercolors. The live image surfaces or sub-surfaces colored in the mentionedkey color—for example green—will then be replaced with the virtualrepresentation of the machine surroundings and/or the visible machinecomponents, so that the superimposed image or the superimposedrepresentation shows the control station of the simulator, itscomponents and body parts of the user situated in the field of vision ofthe live camera in real as a live image on the one hand, and on theother hand the virtual representation of the machine surroundings andthe machine components visible therein in the window areas of theoperator's cab wall captured by the live camera.

The mentioned virtual representation of the machine surroundings canadvantageously be changed by the graphical simulation module and beadapted to different scenarios depending on different data sets, whichcan be imported into the simulation module via an interface. Inparticular, planning data such as CAD data of a building to beconstructed and/or construction site data, which, depending on theconstruction progress, show the actual state of the constructedbuilding, can be imported into the simulation module via a correspondingdata interface and be used by the simulation module to generate thevirtual representation of the machine surroundings according to theimported data set, in particular depending on the imported planning dataand/or actual data of the construction site, or adapt it thereto.

Linking the graphical simulation module with construction site orbuilding information allows using the simulator in a targeted manner forthe training of the works to be done for a certain building or a certainconstruction site. If, for example, a complicated crane lifting movementis to be performed, which is to maneuver a load past various obstaclesand must place a load in a zone of the building that is not visible,this can be trained at the simulator repetitively.

The mentioned construction or building information can be CAD data orother geometrical data of the building or of the construction site,wherein possibly also image data can be used, which represent the actualbuilding and its construction progress. Such image data can be importedinto the graphical simulation module as data of the machine surroundingsvia the mentioned CAD interface or a suitable image data interface,which module will thereupon adapt the virtual representation to thereceived CAD and/or image data.

Modeling a planned or already existing or partially performedconstruction site and the corresponding generation of the virtualrepresentation of the machine surroundings by the graphical simulationmodule is, in particular, also a valuable aid to ensure logistics at theconstruction site and be able to simulate and train critical processeseven before the start of construction.

In a further development of the invention, provision can be made for theuse of the simulator as a remote controller for the remote-controllingof a “real” crane, construction machine or industrial truck, whereinadvantageously, a communication link can be provided between the crane,the construction machine and/or the industrial truck on the one hand,and the simulator on the other hand, via which link control commandsinput at the control station of the simulator can be transmitted to thecontrol device of the crane, the construction machine and/or theindustrial truck. The “real” crane or the respectively remote-controlled“real” device performs the control commands input at the control stationof the simulator, and at the same time, the virtual representation ofthe crane surroundings and the crane components visible thereingenerated by the simulator shows how the crane implements the controlcommands. Here, it may be provided to feed the movement parameters andsensor signals detected at the real crane back to the simulator and touse them there for generating the virtual representation of the cranesurroundings in order to ensure that actually a representationcorresponding to the actual crane surroundings and position is beingdisplayed on the display device of the simulator.

According to another object of the present invention, a graphicalsimulation module for a simulator for a machine is provided, thegraphical simulation module for calculating a virtual representation ofmachine surroundings visible from a control station of the simulator,the graphical simulation module comprising a data interface configuredto import data of a real building and/or real construction site, and animage processing device configured to generate and/or adjust the virtualrepresentation of the machine surroundings depending on the importeddata of the real building and/or real construction site.

According to another object of the present invention, a graphicalsimulation module for a simulator for a machine is provided, thegraphical simulation module for calculating a virtual representation ofmachine surroundings visible from a control station of the simulator,the graphical simulation module comprising a data interface configuredto import data from a server, the data representative of various stagesof construction of a real building at a building site, and an imageprocessing device configured to generate and/or adjust the virtualrepresentation of the machine surroundings depending on the importeddata.

According to another object of the present invention, a simulator for aconstruction machine comprises a movement simulation module fordetermining movements and/or deformations of construction machinecomponents depending on entered control commands, and a control stationcomprising an input means for entering control commands, one of thegraphical simulation modules described above, and a display device fordisplaying the virtual representation, wherein the movement simulationmodule includes a data emulation apparatus for emulating movement dataof the construction machine to be simulated, wherein the data emulationapparatus comprises a first drive unit configured to perform actuatormovements depending on control commands entered at the control station,wherein the graphical simulation module is configured for calculatingthe virtual representation depending on the determined movements and/ordeformations, and wherein the graphical simulation device is furtherconfigured to generate the virtual representation under consideration ofthe performed actuator movement.

The data interface can be a CAD interface.

The data interface can be a CAD interface and the image processingdevice is configured to generate and/or adjust the virtualrepresentation of the machine surroundings depending on the CAD dataimported through the CAD interface, and/or an image data interface andthe image processing device is configured to generate and/or adjust thevirtual representation of the machine surroundings depending on digitalimage data imported via the image data interface.

The image processing device can be further configured to generate and/oradjust the virtual representation of the machine surroundings dependingon the CAD data imported through the CAD interface.

The graphical simulation module can be configured to calculate thevirtual representation of the machine surroundings depending onmovements and/or deformations of machine components determined by amovement simulation module.

The determined movements and/or deformations of the machine componentscan depend upon control commands input via an input means.

The graphical simulation module can be further configured to generatethe virtual representation based upon an actuator movement depending onone or more of the control commands.

Independently of the mentioned option of use as a remote controlstation, the invention will hereinafter be explained in greater detailby means of a preferred exemplary embodiments and associated drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic representation of the components of a cranesimulator according to an advantageous embodiment of the invention,

FIG. 2 is a schematic representation of the hardware components for thedata emulation of the movement simulation module, and

FIG. 3 is a schematic representation, in the type of a circuit diagram,of the overall concept of the simulator from the preceding FIGS. 1-2 andthe functional interaction of the components thereof.

DETAIL DESCRIPTION OF THE INVENTION

To facilitate an understanding of the principles and features of thevarious embodiments of the invention, various illustrative embodimentsare explained below. Although exemplary embodiments of the invention areexplained in detail, it is to be understood that other embodiments arecontemplated. Accordingly, it is not intended that the invention islimited in its scope to the details of construction and arrangement ofcomponents set forth in the following description or illustrated in thedrawings. The invention is capable of other embodiments and of beingpracticed or carried out in various ways.

As used in the specification and the appended Claims, the singular forms“a,” “an” and “the” include plural references unless the context clearlydictates otherwise. For example, reference to a component is intendedalso to include a composition of a plurality of components. Referencesto a composition containing “a” constituent is intended to include otherconstituents in addition to the one named.

In describing exemplary embodiments, terminology will be resorted to forthe sake of clarity. It is intended that each term contemplates itsbroadest meaning as understood by those skilled in the art and includesall technical equivalents that operate in a similar manner to accomplisha similar purpose.

Ranges may be expressed as from “about” or “approximately” or“substantially” one value and/or to “about” or “approximately” or“substantially” another value. When such a range is expressed, otherexemplary embodiments include from the one value and/or to the othervalue.

Similarly, as used herein, “substantially free” of something, or“substantially pure”, and like characterizations, can include both being“at least substantially free” of something, or “at least substantiallypure”, and being “completely free” of something, or “completely pure”.

“Comprising” or “containing” or “including” is meant that at least thenamed compound, element, particle, or method step is present in thecomposition or article or method, but does not exclude the presence ofother compounds, materials, particles, method steps, even if the othersuch compounds, material, particles, method steps have the same functionas what is named.

The characteristics described as defining the various elements of theinvention are intended to be illustrative and not restrictive. Forexample, if the characteristic is a material, the material includes manysuitable materials that would perform the same or a similar function asthe material(s) described herein are intended to be embraced within thescope of the invention. Such other materials not described herein caninclude, but are not limited to, for example, materials that aredeveloped after the time of the development of the invention.

As shown in FIG. 1, the simulator 1 can be configured as a cranesimulator, which includes a control station 2 in the form of a craneoperator's cab, which essentially corresponds to a “real” craneoperator's cab that can be used in a crane, for example a tower slewingcrane, a dockside crane, or a maritime crane, or a mobile telescopiccrane.

The mentioned control station 2 can include, in a manner known per se,an operator seat 21, e.g. in the form of an operator chair 20, aroundwhich several input means 18 for inputting control commands arearranged. The mentioned input means 18 can, for example, comprises ajoystick, a touchscreen, a control lever, input buttons and witches,rotary controllers, shift controllers, etc.

The operator's location is surrounded by a wall 22 of the operator'scab, which can correspond to a cab housing and may comprises windowregions 23, which, in real crane operator's cabs, are made of glass, butin the present case are colored in a certain color, for example coatedwith a green foil, in order to be able to fade-in a virtual machinesurroundings using green-screen technique, as will be explained later.

The control station 2 is mounted on a movement platform 7, by means ofwhich the control station 2 can be moved along multiple axes. Here,advantageously, the movement platform is formed to be moveable alongmultiple axes, in particular tiltable or rotatable along all threespatial axes x, y, and z and translatory displaceable along these axes.

Here, the movement axes x, y and z of the movement platform 7 areassigned to actuators of a drive apparatus 8, for example in the form orelectric motors and/or hydraulic cylinders and/or hydraulic motors, inorder to be able to move the control station 2 around or along thementioned axes.

Here, the drive apparatus 8 is controlled by a movement control device24, which can be realized by an industrial personal computer (IPC).

The mentioned movement control device 24 can in particular be part of amovement simulation module 10, by means of which crane movements and/orpositions and/or orientations of crane components such as of the boom orthe tower and also twists of structural components such as the boom orjob or the tower can be determined depending on the respective controlcommands input at the control station 2. The mentioned movementsimulation module 10 determines, so to say, the effects of the inputcontrol commands on the crane to be simulated, i.e. which movements,positions, orientations and twists of the crane components would resulton the crane to be simulated due to the input control commands, andoutputs corresponding characterizing movement signals characterizing thementioned values.

The mentioned movement simulation module 10 determined the movementvalues not or not completely by computation based upon a calculationscheme, but uses actual hardware components in the form of drive andcontrol components, which perform actual movements and simulate thecorresponding hardware components at a real crane.

As shown by FIGS. 2 and 3 in greater detail, the movement simulationmodule 10 includes the essential components of a crane control 25, justlike can be realized in the control cabinet of a crane. In particular,this crane control 25 includes the frequency converter 15 of differentcrane drives, for example of the rotary mechanism, the trolley drive,and the lifting unit. The mentioned crane control 25 may possiblyinclude further control and/or power electronics components, inparticular load monitoring components, components in regard to the areasof limited operability, etc.

The crane control 25 is connected to the control station 2 and the inputmeans 18 thereof in a communicating manner, so that the crane control 25can further process the input control commands, wherein in particularthe frequency converters 15 control drive units 12, for example in theform of servo drives, depending in the input control commands. Thecontrol commands input at the control station 2 are thus translated intoreal movements or driving moments and forces of the drive units 12.

The mentioned drive units 12 can be coupled with counter drive units 14here, by means of which movements resistance can be added to the driveunits 12, in order to be able to simulate real resistance such aslifting loads, wind power, inertia, od dynamic loads. These counterdrive units 14 can be controlled by the above-mentioned IPC, which alsorealizes the movement control device 24. Here, the control of thecounter drive units 24 can be affected based upon several presets orprograms, for example through predefined lifting loads, predefined windprograms, or by means of predetermined functions or tables with dynamicreactions when decelerating the trolley or the rotary movement. To thatend, corresponding modes, tables, or functions can be stored in astorage module of the controller for the control of the counter driveunits 14.

As shown in FIGS. 2 and 3, the drive units 12 are assigned detectiondevices 13, for example in the form or rotary encoders or other positionand/or movement sensors, by means of which movement or position signalscan be provided, which characterize the actuations of the drive units12. Thus, the movement simulation module 10 provides real sensor signalsas movement parameters, which on the one hand can be displayed at thecontrol station 2, and on the other hand can also be used for furthersimulation functions. In particular, depending on the mentioned movementsignals, that the rotary encoders provide, structural twists such astower bending, boom or jib bending, and similar deformations can bedetermined based upon a calculation scheme, and the device 8 of themovement platform 7 can be controlled to move the control station 2, andthe virtual representation of the crane surroundings can be generated,in each case dependent upon the mentioned sensors signals generated inreality.

As shown in FIG. 3, the movement simulation module 10 may include acomputation unit 11, which in turn can be realized by the abovementioned IPC, by means of which computation unit 11, depending on thecontrol commands input at the control station 2 and/or the emulateddata, the sensor signals generated by the data emulation apparatus 113or the detection device 13 assigned to the drive units 12 can begenerated, structural changes are determined, in particular bending ortorsion in the tower crane and in the boom or jib, wherein thecomputation unit 11 uses a calculation scheme considering structuralrigidity, as explained above.

Based upon the mentioned emulated movement data as well as thedeformation data determined therefrom, the movement control device 24controls the drive device 8 of the movement platform 7 in order to movethe control station 2 and simulate real movements of the craneoperator's cab, which would occur when inputting corresponding controlcommands in a real crane.

On the other hand, the mentioned movement data and, as the case may be,also the mentioned deformation data is used to consider crane reactionsin a virtual representation, which is generated by a graphicalsimulation module 9 and displayed on a display device 3. This virtualrepresentation particularly shows the crane surroundings as well ascrane components visible therein, such as the boom, or the load hook,and may essentially correspond to the image seen by the crane operatorwhen looking out of the window. The mentioned virtual representation maycorrespond to the pixel representation in multiple colors in the form ofa photo or film-like digital image, for example. However, as analternative, a simplified graphical representation can be provided,although a most realistic, photo- or film-like representation image ispreferred.

Advantageously, the mentioned virtual representation of the cranesurroundings and the crane components visible therein are superimposedwith a live image, which shown real components from the control station2, in particular components such as input means 18, the hands or theforearm if the user and other components located in the field of viewfrom the head of the user of the simulator in the viewing direction.

Advantageously, a camera 16 is provided to that end, which can be formedas a head camera worn on the head of the user, and which may comprisecorresponding attachment and/or holding means for attaching it on thehead, for example in the form of a helmet camera. In the case that thedisplay device 3 is formed, advantageously, in the form of virtualreality glasses 4, worn by the user, the camera 16 can be integrated inthese VR glasses.

Advantageously, the camera 16 is formed as a stereoscopic camera, inorder to be able to provide stereoscopic images corresponding to theviewing axes of the two eyes of the user.

The superimposition device 17 for superimposing the virtualrepresentation of the crane surroundings generated by the graphicalsimulation module 9 with the lie image of the camera 16 can inparticular include a color-based image processing module 26, which canoperate in accordance with the green screen technique. In particular,the mentioned color-based image processing module 26 can recognize imageregions within the live image of the camera 16, which have a certaincolor deviating from the remaining image sub-surfaces, and the replacethese image regions by the virtual representation from the simulationmodule 9.

To that end, advantageously, the control station 2 can include anoperator's cab wall 22, in which window regions 23—for examplecorresponding to the inspection windows of a real operator's cab—arecolored in a key color, which most significantly differs from theremaining colors of the remaining components situated in the field ofvision of the camera, such as the color of the window frame, the inputmeans 18 and the clothes and skin color of the operator, so that thelive image captured in the control station 2 shows the mentioned coloredsurfaces in a certain color representation, while all other imagesurfaces are shown in other colors. The live image surfaces orsub-surfaces colored in the mentioned key color—for example green—arethen replaced with the virtual representation of the machinesurroundings and/or the machine components visible therein, which isgenerated by the graphical simulation module 9, so that the superimposedimage or the superimposed representation, on the one hand, shows thecontrol station 2 of the simulator, the components thereof and bodyparts of the user situated in the field of vision of the camera as alive image in reality, and, on the other hand, shows the virtualrepresentation of the machine surroundings and the machine componentsvisible therein in the window regions 23 of the operator's cab wall 22captured by the live camera 16.

The mentioned virtual representation of the machine surroundings canadvantageously be changed by the graphical simulation module 9 and beadapted to various scenarios depending on different data sets, which canbe imported via an interface into the simulation module. In particular,planning data such as CAD data of a building to be constructed and/oractual data of the construction site and/or image data, which show theactual state of the constructed building depending on the progress ofconstruction, can be imported into the simulation module 9 via acorresponding data interface, for example a CAD interface and/or animage data interface, and be used by the simulation module 9 to generatethe virtual representation of the machine surroundings in accordancewith the imported data set, in particular dependent upon the importedplanning data and/or actual data of the construction site, or adapt itthereto.

Numerous characteristics and advantages have been set forth in theforegoing description, together with details of structure and function.While the invention has been disclosed in several forms, it will beapparent to those skilled in the art that many modifications, additions,and deletions, especially in matters of shape, size, and arrangement ofparts, can be made therein without departing from the spirit and scopeof the invention and its equivalents as set forth in the followingclaims. Therefore, other modifications or embodiments as may besuggested by the teachings herein are particularly reserved as they fallwithin the breadth and scope of the claims here appended.

We claim:
 1. A graphical simulation module for a simulator for amachine, the graphical simulation module for calculating a virtualrepresentation of machine surroundings visible from a control station ofthe simulator, the graphical simulation module comprising: a datainterface configured to import data of a real building and/or realconstruction site; and an image processing device configured to generateand/or adjust the virtual representation of the machine surroundingsdepending on the imported data of the real building and/or realconstruction site.
 2. The graphical simulation module of claim 1,wherein the data interface is a CAD interface.
 3. The graphicalsimulation module of claim 1, wherein the data interface is one or bothof: a CAD interface and the image processing device is configured togenerate and/or adjust the virtual representation of the machinesurroundings depending on the CAD data imported through the CADinterface; and an image data interface and the image processing deviceis configured to generate and/or adjust the virtual representation ofthe machine surroundings depending on digital image data imported viathe image data interface.
 4. The graphical simulation module of claim 2,wherein the image processing device is further configured to generateand/or adjust the virtual representation of the machine surroundingsdepending on the CAD data imported through the CAD interface.
 5. Thegraphical simulation module of claim 4, wherein the graphical simulationmodule is configured to calculate the virtual representation of themachine surroundings depending on movements and/or deformations ofmachine components determined by a movement simulation module.
 6. Thegraphical simulation module of claim 5, wherein the determined movementsand/or deformations of the machine components depend upon controlcommands input via an input means.
 7. The graphical simulation module ofclaim 6, wherein the graphical simulation module is further configuredto generate the virtual representation based upon an actuator movementdepending on one or more of the control commands.
 8. A graphicalsimulation module for a simulator for a machine, the graphicalsimulation module for calculating a virtual representation of machinesurroundings visible from a control station of the simulator, thegraphical simulation module comprising: a data interface configured toimport data from a server, the data representative of various stages ofconstruction of a real building at a building site; and an imageprocessing device configured to generate and/or adjust the virtualrepresentation of the machine surroundings depending on the importeddata.
 9. The graphical simulation module of claim 8, wherein the datainterface is a CAD interface.
 10. The graphical simulation module ofclaim 8, wherein the data interface is one or both of: a CAD interfaceand the image processing device is configured to generate and/or adjustthe virtual representation of the machine surroundings depending on theCAD data imported through the CAD interface; and an image data interfaceand the image processing device is configured to generate and/or adjustthe virtual representation of the machine surroundings depending ondigital image data imported via the image data interface.
 11. Thegraphical simulation module of claim 9, wherein the image processingdevice is further configured to generate and/or adjust the virtualrepresentation of the machine surroundings depending on the CAD dataimported through the CAD interface.
 12. A simulator for a constructionmachine comprising: a movement simulation module for determiningmovements and/or deformations of construction machine componentsdepending on entered control commands; and a control station comprising:an input means for entering control commands; the graphical simulationmodule of claim 8; and a display device for displaying the virtualrepresentation; wherein the movement simulation module includes a dataemulation apparatus for emulating movement data of the constructionmachine to be simulated; wherein the data emulation apparatus comprisesa first drive unit configured to perform actuator movements depending oncontrol commands entered at the control station; wherein the graphicalsimulation module is configured for calculating the virtualrepresentation depending on the determined movements and/ordeformations; and wherein the graphical simulation device is furtherconfigured to generate the virtual representation under consideration ofthe performed actuator movement.
 13. The simulator of claim 12, whereinthe data interface is a CAD interface.
 14. The simulator of claim 12,wherein the data interface is one or both of: a CAD interface and theimage processing device is configured to generate and/or adjust thevirtual representation of the machine surroundings depending on the CADdata imported through the CAD interface; and an image data interface andthe image processing device is configured to generate and/or adjust thevirtual representation of the machine surroundings depending on digitalimage data imported via the image data interface.
 15. The simulator ofclaim 13, wherein the image processing device is further configured togenerate and/or adjust the virtual representation of the machinesurroundings depending on the CAD data imported through the CADinterface.
 16. The simulator of claim 12 further comprising: a driveapparatus; a detection device of the emulation apparatus; and a seconddrive unit of the movement simulation module; wherein the controlstation is moveably mounted to the drive apparatus for moving thecontrol station depending on the movements and/or deformationsdetermined by the movement simulation module; wherein detection deviceconfigured to: detect movements of the first drive unit; and provide amovement and/or position signal; wherein the second drive unit iscoupled with the first drive unit, the second drive unit configured toexert a counter moment and/or a counter load in order to simulateactually occurring loads, resistances or inertia; and wherein themovement simulation module is configured to determine movements and/ordeformations of machine components depending on at least a portion ofthe entered control commands.